Rescoring Sequence-to-Sequence Models for Text Line Recognition with CTC-Prefixes

In contrast to Connectionist Temporal Classification (CTC) approaches, Sequence-To-Sequence (S2S) models for Handwritten Text Recognition (HTR) suffer from errors such as skipped or repeated words which often occur at the end of a sequence. this paper, combine best both we propose use CTC-Prefix-Score during S2S decoding. Hereby, beam search, paths that are invalid according CTC confidence matrix penalised. Our network architecture is composed Convolutional Neural Network (CNN) visual backbone, bidirectional Long-Short-Term-Memory-Cells (LSTMs) encoder, and decoder Transformer with inserted mutual attention layers. The confidences computed on encoder while only used character-wise We evaluate setup three HTR data sets: IAM, Rimes, StAZH. On achieve competitive Character Error Rate (CER) 2.95% when pretraining our model synthetic including character-based language contemporary English. Compared other state-of-the-art requires about 10–20 times less parameters. Access shared implementations via link GitHub .